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A METHOD OF RESTORING PARTIALS OF A SOUND SIGNAL 

The present invention relates to the field of 
telecommunications and in particular to the field of 
digitally processing a sound signal and to the harmonic 
5 representation of a sound signal. 

In harmonic modeling of digital audio signals, the 
sound signal is represented by a set of oscillators whose 
parameters (frequency, amplitude, phase) vary slowly over 
time. The harmonic analysis comprises short-term 

10 time/frequency analysis for determining the values of 

these parameters followed by extraction of peaks and then 
tracking of partials. 

The signal to be modeled is divided into frames of £ 
samples (typically ^=1024). A short-term time/frequency 

15 analysis module (which typically executes a Fourier 
transform) calculates the short-term spectrum of the 
signal for each frame. A module for extracting peaks 
retains only the peaks that are the most pertinent a 
priori, one criterion being keeping only the highest 

20 energy peaks, for example. A third and final module 
attempts to link the peaks with each other over time, 
i.e. from one frame to another, to form the partials. 
During its life cycle each partial corresponds to one 
oscillator. 

25 That type of analysis and representation may be used 

in particular during bit rate reduction coding, 
parametric coding (which processes three aspects of the 
signal: transients, sinusoids, noise), separation and 
indexing of sound sources, and restoration of sound 

30 files. 

It is accepted at present that the best quality is 
achieved when synthesizing partials by using phase 
interpolation techniques proposed by Robert J. McAulay 
and Thomas F. Quatieri in the paper "Speech 
35 Analysis/Synthesis Based on a Sinusoidal Representation", 
IEEE Transactions on Acoustics, Speech and Signal 
Processing, pp. 744-754, 1986, or by Laurent Girin, 
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Sylvain Marchand, Joseph di Martino, Axel Robel and 
Geoffroy Peeters in the paper "Comparing the order of a 
Polynomial Phase Model for the Synthesis of Quasi- 
Harmonic Audio Signals", WASPAA, New Paltz, NY, USA, 

5 October 2003. Those techniques are used to synthesize a 
partial from a peak (4,/,^ f ) to a peak (A. +x ,f M9 q> M ) by 

calculating all the intermediate phases using third or 

fifth order polynomials, the frequencies being deduced by 

derivation. Third order interpolation is used when only 

10 the start and end frequencies and phases are known. 

Fifth order interpolation is used when the second order 

variations of the phase are also known (these are 

equivalent to first order variations of frequency since 

by definition frequency is the derivative of phase) . 
15 Synthesizing a partial between the peaks P i {A i ^f i ,(p i ) and 

consists in calculating the values p(n) of 

the partial between the frames i and i+1: 

Pi (n) = p(li + n) = A. (n) cos(#>. (n)) , n = 0, t - 1 ( 1 ) 

To this end, it is known in the art to calculate all 
20 the intermediate phases using one of the following two 
interpolation methods . 

For third order interpolation according to McAulay, 
the phase is calculated from the following expression, in 
which Te is the sampling period: 
2 5 <p. (n) = p f + 2n fpTe + a (nTef + /? (nTef ( 2 ) 

The two unknowns a and fi are calculated by solving 
a system of equations in (f n <P n ff+xiVi+x) • The frequencies 
are deduced by differentiation: 

2n fi{n) = 2nfi + 2anTe + 3 p (nTef ( 3 ) 

30 For fifth order interpolation according to Girin et 

al., the first order variations 8f i and Sf i+l of the 

frequency at the peaks P f and P i+X are assumed to be known. 

The phase is then calculated from the following 
expression : 

35 (pX^^^+lnfpTe + ^inTef +p(nTef + y (nTef +S (nTef (4) 
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The three unknowns fl,S,y are calculated by solving a 
system of equations in {f n f M *9 n 9 M * s ft* s fM) • The 
frequencies are deduced by differentiation: 

2*/J (w) = 2*/J + £ f^Te + 3/3 (nTef + Ay{nTef + 55 (nTe) 4 ( 5 ) 

5 For various reasons, it may happen that certain 

partials in the signal are absent, corrupted, or 
discontinuous at the end of analysis and/or at the 
beginning of synthesis. For example, they may be absent 
at the input of the decoder in an Internet sound program 

10 broadcast application in the event of loss of packets, 
they may be corrupted if the signal to be analyzed is 
suffering interference from an unwanted signal (noise, 
click, other signal, etc.), or they may be discontinuous 
if their energy is too low for them to be correctly 

15 detected continuously. To create a synthesized signal as 
close as possible to the original signal it is then 
necessary to restore the missing peaks. This 
necessitates creating peaks each characterized by an 
amplitude, a frequency, and a phase. 

20 The above prior art interpolation techniques are 

used to synthesize the portions corresponding to the 
missing peaks and to restore the partials. 

However, those prior art interpolation techniques 
are adapted to use in the short-term, i.e. over a period 

25 of less than 10 milliseconds (ms) . For longer periods, 
the re-synthesized signal is often very different from 
the original signal and disagreeable artifacts may 
appear. Those techniques ensure phase continuity between 
the existing peaks and the restored peaks but are not 

30 able to control the induced frequencies resulting from 
equations (3) and (5) . That effect increases in direct 
proportion to the interpolation distance. 

One object of the invention is to propose an 
alternative solution to the problem of restoring a 

35 missing portion identified as that of a partial, in 

particular if the missing portion corresponds to a long 
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period (greater than 10 ms) , for which the prior art 

techniques are relatively ineffective. 

Accordingly, the technical problem to be solved by 

the present invention is that of proposing a method of 

5 restoring missing portions of partials of a sound signal 

during harmonic analysis in which the sound signal is 

divided into time frames to which time/frequency analysis 

is applied that supplies successive short-term spectra 

represented by sample frequency frames, the analysis 

10 further consisting in extracting spectrum peaks in the 

frequency frames and linking them together over time to 

form partials, this method being an alternative to the 

prior art solutions. 

In accordance with the present invention, one 

15 solution to the stated technical problem consists in that 
said method of restoring a partial between a peak P. and a 

peak P i+N whose frequency co and phase are known is 

characterized in that it comprises the steps of: 

• estimating the frequency & of each of the missing 
20 peaks P M to P i+N ^of this partial; 

• calculating the phase <p from peak to peak, from 
the phase of the peak P. to that of the peak P i+N , for all 

the frequencies co previously estimated; 

• calculating the phase error errcp between the 

25 calculated phase <p and the known phase at the same peak 
P 

• correcting each calculated phase <p by a value that 
is a function of the phase error errcp . 

A method of the invention differs from the prior art 
30 methods in that it offers finer control of the frequency 
of the missing peaks and subsequent calculation of the 
corresponding phases to ensure continuity with the phases 
of the existing peaks. Accordingly, a method of the 
invention re-synthesizes signals corresponding to the 
35 missing partial portions without artifacts, in contrast 
to the prior art methods described above. 
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A method of the invention also has the advantage of 
reconstructing a signal that is closer, in terms of the 
reconstruction error, to the original signal than is the 
signal obtained by the prior art methods. 
5 Finally, a method of the invention has the advantage 

of using an algorithm of low complexity* 

The invention further consists in a synthesizer for 

synthesizing a sound signal for implementing a method of 
restoring a partial between a peak P { and a peak P i+N , for 

10 example an audio decoder or a parametric coder adapted to 
use a method of the invention. 

The invention further consists in a computer program 
product loadable directly into the internal memory of the 
above synthesizer or group of synthesizers and comprising 

15 software code portions for executing steps of a method 
according to the invention when the program is executed 
on the synthesizer or group of synthesizers. 

The invention further consists in a medium usable in 
the above synthesizer or group of synthesizers on which 

20 there is stored a computer program product loadable 

directly into the internal memory of the synthesizer or 
group of synthesizers and comprising software code 
portions for executing steps of a method according to the 
invention when the program is executed on the synthesizer 

25 or group of synthesizers. 

Other features and advantages of the invention 
become apparent in the course of the following 
description, which is given with reference to the 
appended drawing, which is provided by way of 

30 non-limiting example. 

Figure 1 is a flowchart of one example of the 
invention . 

Figure 2 is a diagram of one example of the use of a 
method of the invention. 
35 A method 1 of the invention proceeds in the 

following manner, described here with reference to the 
Figure 1 flowchart. The method consists in restoring a 
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partial between a peak P t and a peak P j+N whose frequencies 
co and phases q> are known. 

In a first step 2, the method estimates the 

frequency co and the amplitude A of each of the missing 
5 peaks P i+l to P i+N _ } , for example by linear prediction or 

interpolation methods known in the art. 

Consider a partial consisting of a succession of 
linked peaks P i {^A iy (O n cp i ) known at times iT and 

characterized by: 
10 A i , the amplitude of the peak at the time iT; 

co i , the frequency of the peak at the time iT; and 

g>. , the phase of the peak at the time iT, modulo 2n. 

The frequency of the missing peaks between the peaks 
P. and P i+N is estimated by means of linear interpolation 

15 between co i and co UN , for example, or linear past or future 

prediction, as described in the paper "Enhanced Partial 

Tracking using linear Prediction", Mathieu Lagrange, 

Sylvain Marchand, Martin Raspaud and Jean-Bernard Rault, 

Proceedings of the Digital Audio Effects (DAFx) 

20 Conference, ppl41-146, Queen Mary College, University of 

London, UK, September 2003, for example, or by means of a 

weighted past or future combination. 

The amplitude A of. the missing peaks is estimated by 
linear interpolation between A. and A i+N , for example, 

25 linear past or future prediction, or weighted past or 
future combination . 

In a second step 3, the method calculates the phase 
q> from peak to peak, from the phase of the peak /?to that 

of the peak P i+N . This calculation is effected for each 

30 of the frequencies co previously estimated. 

Let ^. and co i be the starting phase and frequency and 

{^ l+ i,..- 5 ^ l+ ^_i} estimated frequencies in the range to be 
reconstructed. To extend the partial between the peak P i 
and the peak P j+N the phase is calculated from the 

35 following expression: 
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To avoid generating discontinuities that would 
compromise the quality of the re-synthesis, it is 
necessary to obtain at the time i+N a reconstructed phase 
5 (p i+N equal to <p i+N . The data in the above expression (6) 

being either approximate or predicted, it is 
statistically impossible to obtain this equality. 
Consequently, the subsequent steps of the method divide 
the phase error errcp calculated at the time i+N between 
10 all the missing peaks P M to P i+N _ x previously 

reconstructed . 

In a third step 4, the method calculates the phase 
error errcp between the calculated phase <p i+N and the known 

phase tp j+N at the same peak P i+N . This calculation may use 

15 the following system of equations: 

if \(p i+N -<p i+N +27t\<\<p i+N -(p i+N \ , errcp = cp i + N -% +N +2n (7) 

if \p i+N -<p i+N -27r\<\<p i+N -p i+N \ , err<p = <p i+N -<p i+N -27r (8) 
else err<p = (p i+N -<p i+N (9) 

In a fourth step 5, the method corrects each 
20 calculated phase <p i+n by a value that is a function of the 

phase error errcp . The phase error calculated at the time 
i+N is typically divided uniformly between the calculated 
phases in accordance with the following expression: 

( ~ n \ 

<pcorrected i+n = mod <p^ n + errcp — ,2k n = l,...,A r -l (10) 

V N ) 

25 The distribution need not be uniform, and may 

conform to a non-linear law, for example. 

The Figure 2 example of use consists in restoring 
partials by means of the method 1 of the invention at the 
time of harmonic analysis of a sound signal, for example 

30 during parametric coding. The sound signal s (n) is 
represented by a set of oscillators whose parameters 
(frequency, amplitude) vary slowly over time. In the 
conventional way, the harmonic analysis includes 
short-term time/frequency analysis 6 for determining the 
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values of these parameters, followed by extraction of 
peaks 7 followed by tracking 8 of partials. Detection 9 
of gaps in the partials precedes restoring partials by 
the method 1 of the invention. The peaks 



5 Pi+n [A+n'&i+n'&i+n ) reconstructed by executing the method 1 are 

then treated as peaks resulting from the harmonic 
analysis and additive synthesis 10 of the signal 
corresponding to the partial restored from these 
reconstructed peaks may be effected by one of the prior 
10 art (third or fifth order) phase interpolation methods, 
for example . 




